Matthew Palmer: How to deal with the "package gcc: " problem in Puppet
In a comment to yesterday s post on why your Puppet module
sucks, Warren asks
what can be done about the problem of multiple modules needing to include
the same package (Put your package into a class in your module
Absofrickinglutely useless, because the elementary
gcc
being the most common example I m yet to come
across).
As I stated
previously, there
is no sane, standardised way for multiple independent modules to cooperate
to ensure that certain packages are installed for their cooperative use. If
it wasn t already obvious, I consider this to be a significant failing in
Puppet, and one which essentially renders any attempt at public, wide-scale
reuse of Puppet modules impossible. Software packages are such a
fundamental part of system management today that it is rare for a module not
to want to interact with them in some way.
Without strong leadership from Puppet Labs, or someone core to the project
who s willing to be very noisy and obnoxious on the subject, the issue is
fundamentally unsolveable, because it requires either deep (and
non-backwards-compatible) changes to how Puppet works at a core level, or
it needs cooperation from everyone who writes Puppet modules for public
consumption (to use a single, coordinated approach to common packages).
The approaches that I ve seen people advocate using, or that I ve considered
myself, roughly fall into the following camps.
Put your package into a class in your module
Absofrickinglutely useless, because the elementary package
resources you
end up creating will still conflict with other modules package
resources.
Anyone who suggests this can be permanently ignored, as they don t
understand the problem being solved.
Use a globally common module of classes representing common packages
This could work, if you could get everyone who writes modules to use it
(see someone willing to be very noisy and obnoxious ). Practically
speaking, anyone who suggests this doesn t understand human nature. I don t
see this actually happening any time soon.
Use the defined()
function everywhere
By wrapping all your package
resources in if !defined(Package["foo"])
blocks, you can at least stop your module from causing explosions. What
it doesn t do is make sure that the various definitions of the package
resource are congruent (imagine if one was package "gcc": ensure =>
absent
). In order to safely avoid explosion, everyone would
have to follow this approach, which (in the worst case) reduces the problem
to the globally common module of classes.
However, it s the least-worst approach I can practically consider. At least
your module won t be the cause of explosions, and realistically no
module s going to ask for gcc
to be removed (or, if they are, you ll
quickly find them and kill them).
I ve seen calls from Puppet core dev team members to deprecate and remove
the defined()
function entirely. Given the complete lack of alternatives,
this is a disturbing illustration of just how out of touch they are from the
unfortunate realities of practical Puppetry.
Use virtual resources
This is yet another variant of the common class technique that would
require everyone, everywhere, to dance to the same tune. It has all the
same problems, and hence gets a big thumbs-down from me.
Use singleton_packages
This module is a valiant attempt to work around the problem, but practically
speaking is no better than using defined()
because again, if a module
that isn t using singleton_packages
specifies package "gcc": ensure
=> absent
, you re going to end up with assplosion. The mandatory
dependency on hiera
doesn t win me over, either (I m a hiera-skeptic, for
reasons which I might go into some other time).
Use only modules from a single source
This is the solution that I use myself, if that s any recommendation
As a result of pretty much every publically-available Puppet module sucking
for one reason or
another, I do not,
currently, have any externally-written Puppet modules in the trees that I
use. Every module (158, at current count) has been written by someone in
the sysadmin team at $DAYJOB, to our common standards. This means that I
can coordinate use of package
resources, using our module of common
classes where appropriate, or refactor modules to separate concerns
appropriately.
If you re wondering why we don t have all of these 158 modules available
publically, well, we have a few of them up, but
yes, the vast majority of them aren t publically available. Some of them
suck mightily,
while many others are just too intertwined with other modules to be able to
use on their own, and we don t want to release crap code there s far too
much of that out there already.
Use the defined()
function everywhere
By wrapping all your package
resources in if !defined(Package["foo"])
blocks, you can at least stop your module from causing explosions. What
it doesn t do is make sure that the various definitions of the package
resource are congruent (imagine if one was package "gcc": ensure =>
absent
). In order to safely avoid explosion, everyone would
have to follow this approach, which (in the worst case) reduces the problem
to the globally common module of classes.
However, it s the least-worst approach I can practically consider. At least
your module won t be the cause of explosions, and realistically no
module s going to ask for gcc
to be removed (or, if they are, you ll
quickly find them and kill them).
I ve seen calls from Puppet core dev team members to deprecate and remove
the defined()
function entirely. Given the complete lack of alternatives,
this is a disturbing illustration of just how out of touch they are from the
unfortunate realities of practical Puppetry.
Use virtual resources
This is yet another variant of the common class technique that would
require everyone, everywhere, to dance to the same tune. It has all the
same problems, and hence gets a big thumbs-down from me.
Use singleton_packages
This module is a valiant attempt to work around the problem, but practically
speaking is no better than using defined()
because again, if a module
that isn t using singleton_packages
specifies package "gcc": ensure
=> absent
, you re going to end up with assplosion. The mandatory
dependency on hiera
doesn t win me over, either (I m a hiera-skeptic, for
reasons which I might go into some other time).
Use only modules from a single source
This is the solution that I use myself, if that s any recommendation
As a result of pretty much every publically-available Puppet module sucking
for one reason or
another, I do not,
currently, have any externally-written Puppet modules in the trees that I
use. Every module (158, at current count) has been written by someone in
the sysadmin team at $DAYJOB, to our common standards. This means that I
can coordinate use of package
resources, using our module of common
classes where appropriate, or refactor modules to separate concerns
appropriately.
If you re wondering why we don t have all of these 158 modules available
publically, well, we have a few of them up, but
yes, the vast majority of them aren t publically available. Some of them
suck mightily,
while many others are just too intertwined with other modules to be able to
use on their own, and we don t want to release crap code there s far too
much of that out there already.
Use singleton_packages
This module is a valiant attempt to work around the problem, but practically
speaking is no better than using defined()
because again, if a module
that isn t using singleton_packages
specifies package "gcc": ensure
=> absent
, you re going to end up with assplosion. The mandatory
dependency on hiera
doesn t win me over, either (I m a hiera-skeptic, for
reasons which I might go into some other time).
Use only modules from a single source
This is the solution that I use myself, if that s any recommendation
As a result of pretty much every publically-available Puppet module sucking
for one reason or
another, I do not,
currently, have any externally-written Puppet modules in the trees that I
use. Every module (158, at current count) has been written by someone in
the sysadmin team at $DAYJOB, to our common standards. This means that I
can coordinate use of package
resources, using our module of common
classes where appropriate, or refactor modules to separate concerns
appropriately.
If you re wondering why we don t have all of these 158 modules available
publically, well, we have a few of them up, but
yes, the vast majority of them aren t publically available. Some of them
suck mightily,
while many others are just too intertwined with other modules to be able to
use on their own, and we don t want to release crap code there s far too
much of that out there already.
package
resources, using our module of common
classes where appropriate, or refactor modules to separate concerns
appropriately.
If you re wondering why we don t have all of these 158 modules available
publically, well, we have a few of them up, but
yes, the vast majority of them aren t publically available. Some of them
suck mightily,
while many others are just too intertwined with other modules to be able to
use on their own, and we don t want to release crap code there s far too
much of that out there already.